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DETAILED ACTION 

Response to Arguments 

1. Applicant's arguments with respect to claims 1, 20, 49-51, and 98 have been considered 
but are moot in view of the new ground(s) of rejection. 

Claim Rejections - 35 USC §103 

2. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

3. Claims 1-3, 20-22, 50-53, 70-71, and 98 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Cornog et al. (US 6,570,624 B2) in view of Wilensky et al (US 6,721,446). 

Re claims 1-3, 20-22, 50-53, 70-71, and 98, Cornog discloses a method for masking (fig. 
1) a foreground portion from a background portion of a video (FG1 and FG2 of fig. 4 A) , the 
method comprising: receiving a first input (100 of fig. 1) defining a first border region (FG1, 
FIRST IMAGE (A) of fig. 4 A) that includes a border between a foreground portion (FG1 of fig. 
4A) and background portion (BG of fig. 4A) of a first image (FIRST IMAGE (A) of fig. 4 A), 
the first digital image being one of a sequence of images defining a digital video, the first 
border region (FG1 must has a border between the foreground FG1 and BG of the first image) 
further including only a part of the foreground portion and only a part of the background 
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portion; receiving a second input (102 of fig. 1) defining a second border region (FG2 of fig. 
4A) that includes a border (the foreground has a border between foreground and background 
BG) in a second digital image (SECOND IMAGE (B) of fig. 4A) 5 the second image being 
another one of the sequence of digital images, the second border region further including only a 
part of the foreground portion and only a part of the background portion (FG2 and BG of fig. 
4A); interpolating (1 14 of fig. 1, e.g. the first image and the second image are blended together 
called interpolating process) between the first and second border regions to define an 
intermediary border region (COMBINATION MAP of fig. 4B, e.g. the first border of the first 
image and the second border of the second image are combined, interpolated to form the 
intermediate border) for an image intermediary in the sequence to the first and second digital 
images (fig. 4B; See also col. 13, line 29-col.l4, line 13); and using the first, second, and 
intermediary border regions for (1 16 of fig. 1) masking the foreground portion from the 
background portion in the digital video (fig. 4B; see also col. 14, lines 14-37); wherein 
receiving a first and a second input (100 and 102 of fig. 1) comprise: receiving user inputs (100 
and 102 of fig. 1; 1002 and 1004 of fig. 10) defining the border regions; and where interpolating 
an intermediary border region (1012 of fig. 10) comprises: interpolating (1 14 of fig. 1) an 
intermediary border region automatically without user input; wherein using the first, second, 
and intermediary border regions for masking (col. 14, lines 13-38) the foreground portion from 
the background portion comprises: determining for a pixel in a border region whether it 
includes data that is associated with the foreground portion (510, 512, 514 of fig. 5 A); and using 
the result of the determining step to mask the foreground portion from the background portion 
in the digital video ( fig. 5B). 
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It is noted that Comog does not particularly teach a sequence of digital images that define 
video as claimed. 

However, Wilensky et al. teaches a sequence of digital images that define video used to 
perform the intrinsic color between regions of digital images (figs. 1-12, see also Abstract and 
col. 4, line 45 - col. 5, line49). Therefore, taking the teachings of Cornog and Wilensky as a 
whole, it would have been obvious to one of ordinary skill in the art to incorporate the teachings 
of Wilensky into the method of Cornog for the same purpose of interpolating the first and 
second border regions to define intermediate border regions of the digital video image. Doing 
so would provide the masking and extracting of object from digital images is achieved with 
high accuracy, and multiple object can be extracted from an image in a single step. 

4. Claims 1-4, 7-9, 19, 20, 38, 49-51, 69 and 87 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Chen (US 6,556,704 Bl) in view of Wilensky et al (US 6,721,446). 

Re claims 1, 20, and 49-5 1, Chen teaches a method that is implemented on the computer 
(fig. 12) for masking a foreground portion of a digital image from a background portion of a 
digital image (218 of fig. 6), the digital image being part of a video comprising a time sequence 
of digital images (21 lb, TOP IMAGE, 211a, BOTTOM IMAGE of fig. 6), each image being 
defined by a plurality of pixels (col. 3, fig. 1, e.g. pixel locations), the method comprising: 
receiving a first input defining a first border region (14a of fig. 1), the first border region 
including at least a part of the foreground portion (14a of fig. 1 is a part of the foreground) and 
at least a part of the background portion (15a of fig. 1) in a first digital image (BOTTOM 
IMAGE of fig. 1); receiving a second input defining a second border region (14b and 15b of fig. 
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1), the second border region including at least a part of the foreground portion (14b of fig. 1) and 
at least a part of the background portion (15b of fig. 1) in a second digital image (TOP IMAGE 
of fig. 1); interpolating an intermediary border region for an image intermediary in time to the 
first and second digital images (fig. 1 1, col. 11, line 31 through col. 12, line 30); and using the 
first, second, and intermediary border regions for masking the foreground portion from the 
background portion in the digital video (20 of figs. 1, 2; and 218 of fig. 6; see also col. 10 line 67 
through col. 11, line 30). 

It is noted that Chen does not particularly teach a sequence of digital images that define 
video as claimed. 

However, Wilensky et al. teaches a sequence of digital images that define video used to 
perform the intrinsic color between regions of digital images (figs. 1-12, see also Abstract and 
col. 4, line 45 - col. 5, line49). Therefore, taking the teachings of Chen and Wilensky as a 
whole, it would have been obvious to one of ordinary skill in the art to incorporate the teachings 
of Wilensky into the method of Chen for the same purpose of interpolating the first and second 
border regions to define intermediate border regions of the digital video image. Doing so would 
provide the masking and extracting of object from digital images is achieved with high 
accuracy, and multiple object can be extracted from an image in a single step. 

Re claim 2, Chen further teaches wherein receiving a first and a second input comprise: 
receiving user inputs defining the border regions (116 and 118 of fig. 12); and where 
interpolating an intermediary border region comprises: interpolating an intermediary border 
region automatically without user input (218 of fig. 6, e.g. an algorithm (218) generates 
foreground depth mask formation). 



Application/Control Number: 09/747,945 Page 6 

Art Unit: 2613 

Re claim 3, Chen further teaches determining for a pixel in a border region whether it 
includes data that is associated with the foreground portion (216 of fig. 6, col. 7, line 29 through 
col. 10, line 4); and using the result of the determining step to mask the foreground portion from 
the background portion in the digital video (col. 10, line 8, through col. 11, line 15). 

Re claim 4, Chen further teaches estimating an intrinsic color value for a pixel in the first, 
second, and intermediary border regions (col. 10, lines 6-66); and using the estimated intrinsic 
color value for extracting the foreground portion from the background portion (fig. 9). 

Re claim 7, Chen further teaches wherein receiving inputs indicating the border 
regions comprise: generating a single path having a width encompassing that of 
the border region (812 of fig. 1 1). 

Re claim 8, Chen further teaches wherein generating a single path comprises: generating 
a vector-based single path (col. 10, lines 60-66). 

Re claims 9, Chen further teaches generating a single path having variable thickness 
throughout its length (fig. 1 1). 

Re claims 19, 38, 69 and 87, Chen further teaches wherein the foreground portion is an 
object (14a, 14b of fig. 1). 

5. Claims 1-9, 19, 49, and 50 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Totsuka et al. (US 6,128,046) in view of Cornog et al. (US 6,570,624 B2). 

Re claims 1, Totsuka teaches a method for masking a foreground portion of a digital 
image from a background portion of a digital image, the digital image being part of a video 
comprising a time sequence of digital images, each image being defined by a plurality of pixels 
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(figs. 2A-2C), the method comprising: receiving a first input defining a first border region 
(object at t=tl of fig. 3 A), the first border region including at least a part of the foreground 
portion and at least a part of the background portion in a first digital image (Frame t=tl of fig. 
3A); receiving a second input defining a second border region (Object at t=t2 of fig. 3A), the 
second border region including at least a part of the foreground portion and at least a part of the 
background portion in a second digital image; interpolating an intermediary border region for 
an image intermediary in time to the first and second digital images (Steps S5-1-S5-5 of fig. 
29); and using the first, second, and intermediary border regions for masking the foreground 
portion from the background portion in the digital video (S5-5 of fig. 29). 

It is noted that Totsuka does not particularly teach the interpolating between its frames 
and border regions as claimed. 

However, Cornog teaches receiving a first input (100 of fig. 1) defining a first border 
region (FG1, FIRST IMAGE (A) of fig. 4A) that includes a border between a foreground 
portion (FG1 of fig. 4 A) and background portion (BG of fig. 4 A) of a first image (FIRST 
IMAGE (A) of fig. 4A), the first digital image being one of a sequence of images defining a 
digital video, the first border region (FG1 must has a border between the foreground FG1 and 
BG of the first image) further including only a part of the foreground portion and only a part of 
the background portion; receiving a second input (102 of fig. 1) defining a second border region 
(FG2 of fig. 4A) that includes a border (the foreground has a border between foreground and 
background BG) in a second digital image (SECOND IMAGE (B) of fig. 4A), the second image 
being another one of the sequence of digital images, the second border region further including 
only a part of the foreground portion and only a part of the background portion (FG2 and BG of 
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fig. 4A); interpolating (1 14 of fig. 1, e.g. the first image and the second image are blended 
together called interpolating process) between the first and second border regions to define an 
intermediary border region (COMBINATION MAP of fig. 4B, e.g. the first border of the first 
image and the second border of the second image are combined, interpolated to form the 
intermediate border) for an image intermediary in the sequence to the first and second digital 
images (fig. 4B; See also col. 13, line 29-col.l4, line 13). 

Therefore, taking the teachings of Totsuka and Cornog as a whole, it would have been 
obvious to one of ordinary skill in the art to incorporate the teachings of Cornog into the method 
of Totsuka for the same purpose of interpolating its frames or images. Doing so would reduce 
the time, of operation and the cost of the system. 

Re claims 2, 21, and 22, Totsuka further teaches wherein receiving a first and a second 
input comprise: receiving user inputs defining the border regions (Object segmentation of 
Frame, at t=tl and t=t2 of fig. 3 A); and where interpolating an intermediary border region 
comprises: interpolating an intermediary border region automatically without user input (fig. 
29). 

Re claim 3, Totsuka further teaches determining for a pixel in a border region whether it 
includes data that is associated with the foreground portion (fig. 2 A); and using the result of the 
determining step to mask the foreground portion from the background portion in the digital 
video (fig. 2C). 

Re claims 4 and 23, Totsuka further teaches estimating an intrinsic color value for a pixel 
in the first, second, and intermediary border regions (fig. 12); and using the estimated intrinsic 
color value for extracting the foreground portion from the background portion (fig. 18). 
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Re claims 5 and 26, Totsuka further teaches wherein receiving inputs indicating the 
border regions comprises: generating an inside path located inside a foreground portion (P2 of 
fig. 28B); and generating an outside path located outside the foreground portion and enclosing 
the inside path (PI of fig. 28B), wherein pixels between the inside and outside paths belong to a 
border region (PATH AREA has pixel of fig. 28B). 

Re claims 6 and 27, Totsuka further teaches wherein generating an inside path and an 
outside path comprise: generating a vector-based inside path and a vector-based outside path 
(10 of fig. 30). 

Re claims 7 and 28, Totsuka further teaches wherein receiving inputs indicating the 
border regions comprise: generating a single path having a width encompassing that of the 
border region (PATH segment of fig. 28B). 

Re claims 8 and 27, Totsuka further teaches wherein generating a single path comprises: 
generating a vector-based single path (fig. 28A). 

Re claims 9, Totsuka further teaches generating a single path having variable thickness 
throughout its length (fig. 28B). 

Re claims 19, 38, 69 and 87, Totsuka further teaches wherein the foreground portion is an 
object (fig. 3A). 

Re claims 53-59, see the analysis in claims 1-9. 

6. Claims 1-2, 19-21, 38, 49-52, 69, 87, and 98 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Norton et al. (US 5,912,994) in view of Cornog et al. (US 6,570,624 B2). 
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Re claims 1-2, 19-21, 38, 49-52, 69, 87, and 98, Norton teaches a method that is 
implemented on the computer (figs. 1-6) for masking a foreground portion of a digital image 
from a background portion of a digital image (fig. 2), the digital image being part of a video 
comprising a time sequence of digital images, each image being defined by a plurality of pixels, 
the method comprising: receiving a first input defining a first border region (42 of fig. 2), the 
first border region including at least a part of the foreground portion and at least a part of the 
background portion in a first digital image; receiving a second input defining a second border 
region (42 of fig. 2, more objects of the second frame or image), the second border region 
including at least a part of the foreground portion and at least a part of the background portion in 
a second digital image; interpolating an intermediary border region for an image intermediary in 
time to the first and second digital images (60 of fig. 3); and using the first, second, and 
intermediary border regions for masking the foreground portion from the background portion in 
the digital video (56 of fig. 3); wherein receiving a first and a second input comprise: receiving 
user inputs defining the border regions (fig. 1); and where interpolating an intermediary border 
region comprises: interpolating an intermediary border region automatically without user input 
(12 of fig. l,e.g. the processor has an automatic function to do the interpolating the intermediary 
bolder regions). 

It is noted that Norton does not particularly teach the interpolating between its frames and 
border regions as claimed. 

However, Cornog teaches receiving a first input (100 of fig. 1) defining a first border 
region (FG1, FIRST IMAGE (A) of fig. 4 A) that includes a border between a foreground 
portion (FG1 of fig. 4A) and background portion (BG of fig. 4A) of a first image (FIRST 
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IMAGE (A) of fig. 4A), the first digital image being one of a sequence of images defining a 
digital video, the first border region (FG1 must has a border between the foreground FG1 and 
BG of the first image) further including only a part of the foreground portion and only a part of 
the background portion; receiving a second input (102 of fig. 1) defining a second border region 
(FG2 of fig. 4A) that includes a border (the foreground has a border between foreground and 
background BG) in a second digital image (SECOND IMAGE (B) of fig. 4A), the second image 
being another one of the sequence of digital images, the second border region further including 
only a part of the foreground portion and only a part of the background portion (FG2 and BG of 
fig. 4A); interpolating (1 14 of fig. 1, e.g. the first image and the second image are blended 
together called interpolating process) between the first and second border regions to define an 
intermediary border region (COMBINATION MAP of fig. 4B, e.g. the first border of the first 
image and the second border of the second image are combined, interpolated to form the 
intermediate border) for an image intermediary in the sequence to the first and second digital 
images (fig. 4B; See also col. 13, line 29-col.l4, line 13). 

Therefore, taking the teachings of Norton and Cornog as a whole, it would have been 
obvious to one of ordinary skill in the art to incorporate the teachings of Cornog into the method 
of Norton for the same purpose of interpolating its frames or images. Doing so would reduce 
the time of operation and the cost of the system. 

7. Claims 1-98 are rejected under 35 U.S.C. 103(a) as being unpatentable over Pettigrew et 
al. as shown in figures 1-32, and further in view of Cornog et al. (US 6,570,624 B2). 
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It is noted that Pettigrew does not particularly teach the interpolating between its frames 
and border regions as claimed. 

However, Cornog teaches receiving a first input (100 of fig. 1) defining a first border 
region (FG1, FIRST IMAGE (A) of fig. 4A) that includes a border between a foreground 
portion (FG1 of fig. 4 A) and background portion (BG of fig. 4 A) of a first image (FIRST 
IMAGE (A) of fig. 4A), the first digital image being one of a sequence of images defining a 
digital video, the first border region (FG1 must has a border between the foreground FG1 and 
BG of the first image) further including only a part of the foreground portion and only a part of 
the background portion; receiving a second input (102 of fig. 1) defining a second border region 
(FG2 of fig. 4A) that includes a border (the foreground has a border between foreground and 
background BG) in a second digital image (SECOND IMAGE (B) of fig. 4A), the second image 
being another one of the sequence of digital images, the second border region further including 
only a part of the foreground portion and only a part of the background portion (FG2 and BG of 
fig. 4A); interpolating (1 14 of fig. 1, e.g. the first image and the second image are blended 
together called interpolating process) between the first and second border regions to define an 
intermediary border region (COMBINATION MAP of fig. 4B, e.g. the first border of the first 
image and the second border of the second image are combined, interpolated to form the 
intermediate border) for an image intermediary in the sequence to the first and second digital 
images (fig. 4B; See also col. 13, line 29-col.l4, line 13). Therefore, taking the teachings of 
Pettigrew and Cornog as a whole, it would have been obvious to one of ordinary skill in the art 
to incorporate the teachings of Cornog into the method of Pettigrew for the same purpose of 
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interpolating its frames or images. Doing so would reduce the time of operation and the cost of 
the system. 

Conclusion 

8. Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

Contact Information 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Tung T. Vo whose telephone number is (703) 308-5874. The 
examiner can normally be reached on 6:30 AM - 3:00 PM. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Chris. Kelley can be reached on (703) 305-4856. The fax phone number for the 

organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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